Full-text relevancy ranking

ABSTRACT

A method and system for ranking relevancy of metadata associated with media on a computer network, such as multimedia and streaming media, include categorizing the metadata into sets of metadata. The categories are broad categories relating to areas such as who, what, when, and where, such as artist, media type, and creation date, creation location. Weights are assigned to each set of metadata. Weights are related to technical information such as bit rate, duration, sampling rate, frequency of occurrence of a specific term, etc. A score is calculated for ranking the relevancy of each set of metadata. The score is calculated in accordance with the assigned weight and category. This score is available for search systems (e.g., search engines) and/or users to determine the relative ranking of search results.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from U.S. provisional application No.60/252,273, filed on Nov. 21, 2000, which is herein incorporated byreference in its entirety. This application is related to the followingapplications, each being filed on the same date as the instantapplication: application Ser. No. ______, entitled “Internet StreamingMedia Workflow Architecture,” attorney docket number D4961-00014;application Ser. No. ______, entitled “Interpretive Stream MetadataExtraction,” attorney docket number D4961-00015; application Ser. No.______, entitled “Metadata Quality Improvement,” attorney docket numberD4961-00016; application Ser. No. ______, entitled “Grouping MultimediaAnd Streaming Media Search Results,” attorney docket number D4961-00018;application Ser. No. ______, entitled “Fuzzy Database Retrieval,”attorney docket number D4961-00019; and application Ser. No. ______,entitle “Internet Crawl Seeding,” attorney docket number D4961-00020.

FIELD OF THE INVENTION

The present invention relates to computer related information search andretrieval, and specifically to assessing the relevancy of multimedia andstreaming media utilizing metadata.

BACKGROUND

An aspect of the Internet (also referred to as the World Wide Web, orWeb) that has contributed to its popularity is the plethora ofmultimedia and streaming media files available to users. However,finding a specific multimedia or streaming media file buried among themillions of files on the Web is often an extremely difficult task. Thevolume and variety of informational content available on the web islikely continue to increase at a rather substantial pace. This growth,combined with the highly decentralized nature of the web, createssubstantial difficulty in locating particular informational content.

Streaming media refers to audio, video and interactive files that aredelivered to a user's computer via the Internet or other networkenvironment. One advantage of streaming media is that streaming mediafiles begin to play before the entire file is downloaded, saving usersthe long wait typically associated with downloading the entire file.Digitally recorded music, movies, trailers, news reports, radiobroadcasts and live events have all contributed to an increase instreaming content on the Web. In addition, less expensive high-bandwidthconnections such as cable, DSL and T1 are providing Internet users withspeedier, more reliable access to streaming media content from newsorganizations, Hollywood studios, independent producers, record labelsand even home users themselves.

A user typically uses a search engine to find specific information onthe Internet. A search engine is a set of programs accessible at anetwork site within a network, for example a local area network (LAN) orthe Internet and World Wide Web. One program, called a “robot” or“spider”, pre-traverses a network in search of documents (e.g., webpages) and builds large index files of keywords found in the documents.Typically, a user formulates a query comprising one or more search termsand submits the query to another program of the search engine. Inresponse, the search engine inspects its own index files and displays alist of documents that match the search query, typically as hyperlinks.The user then typically activates one of the hyperlinks to see theinformation contained in the document.

Search engines, however, have drawbacks. For example, many typicalsearch engines are oriented to discover textual information only. Inparticular, they are not well suited for indexing information containedin structured databases (e.g. relational databases), voice relatedinformation, audio related information, multimedia, and streaming media,etc. Also, mixing data from incompatible data sources is difficult forconventional search engines.

Another disadvantage of conventional search engines is that irrelevantinformation is aggregated with relevant information. For example, it isnot uncommon for a search engine on the web to locate hundreds ofthousands of documents in response to a single query. Many of thosedocuments are found because they coincidentally include the same keywordin the search query. Sifting through search results in the thousands,however, is a daunting task. For example, if a user were looking for asong having the title “I Am The Walrus,” the search query wouldtypically contain the word “walrus.” The list of hits would includedocuments providing biological information on walruses, etc. Thus, theuser would have to review an enormous number of these hits beforefinally (if ever) reaching a hit related to the desired song title.Adding to a user's frustration is the possibility that many of thesearch results are duplicates and/or variants of each other, leading tothe same document (e.g. uniform resource locator, URL). Furtherdifficulty occurs in trying to evaluate the relative merit or relevanceof concurrently found documents. The search for specific content basedon a few key words will almost always identify documents whoseindividual relevance is highly variable.

Thus, there is a need for an automated multimedia and streaming mediasearch tool that provides information to a user that overcomes thepreviously described drawbacks and disadvantages.

SUMMARY OF THE INVENTION

A method and system for ranking relevancy of metadata associated withmedia on a computer network include categorizing the metadata into setsof metadata. Each category of metadata includes at least one set ofmetadata. Weights are assigned to each set of metadata. The value ofeach weight is determined in accordance with the content of each set ofmetadata. A score is calculated for ranking the relevancy of each set ofmetadata. The score is calculated in accordance with the assigned weightand category.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other advantages and features of the present inventionwill be better understood from the following detailed description of thepreferred embodiments of the invention, which is provided in connectionwith the accompanying drawings. The various features of the drawings maynot be to scale. Included in the drawing are the following figures:

FIG. 1 is a block diagram of a computer system in accordance with anexemplary embodiment of the present invention;

FIG. 2 is a flow diagram of an exemplary search and retrieval process inaccordance with the present invention;

FIG. 3 is a functional block diagram of an exemplary multimedia and/orstreaming media metadata search, retrieval, and enhancement system inaccordance with the present invention;

FIG. 4 is a flow diagram of an exemplary spider seeding process inaccordance with the present invention;

FIG. 5 is a flow diagram of an exemplary distribution and extractionprocess in accordance with the present invention;

FIG. 6 is a flow diagram of an exemplary interpretive metadataextraction and database retrieval process in accordance with the presentinvention;

FIG. 7 is a flow diagram of an exemplary process for querying databasesin accordance with the present invention;

FIG. 8 is a flow diagram of an exemplary grouping process in accordancewith the present invention;

FIG. 9 is a flow diagram of an exemplary iterative masking process inaccordance with the present invention;

FIG. 10 is a flow diagram of an exemplary metadata quality improvementprocess in accordance with the present invention; and

FIG. 11 is a flow diagram of an exemplary full-text relevancy rankingprocess in accordance with the present invention.

DETAILED DESCRIPTION

Although the invention is described in terms of exemplary embodiments,it is not limited thereto. Rather, the appended claims should beconstrued broadly, to include other variants and embodiments of theinvention, which may be made by those skilled in the art withoutdeparting from the scope and range of equivalents of the invention.

The present invention is a system and method for retrieving media filesand data related to media files on a computer network via a searchsystem utilizing metadata. As used herein, the term “media file”includes audio, video, textual, multimedia data files, and streamingmedia files. Multimedia files comprise any combination of text, image,video, and audio data. Streaming media comprises audio, video,multimedia, textual, and interactive data files that are delivered to auser's computer via the Internet or other communications networkenvironment and begin to play on the user's computer/device beforedelivery of the entire file is completed. One advantage of streamingmedia is that streaming media files begin to play before the entire fileis downloaded, saving users the long wait typically associated withdownloading the entire file. Digitally recorded music, movies, trailers,news reports, radio broadcasts and live events have all contributed toan increase in streaming content on the Web. In addition, the reductionin cost of communications networks through the use of high-bandwidthconnections such as cable, DSL, T1 lines and wireless networks (e.g.,2.5G or 3G based cellular networks) are providing Internet users withspeedier, more reliable access to streaming media content from newsorganizations, Hollywood studios, independent producers, record labelsand even home users themselves.

Examples of streaming media include songs, political speeches, newsbroadcasts, movie trailers, live broadcasts, radio broadcasts, financialconference calls, live concerts, web-cam footage, and other specialevents. Streaming media is encoded in various formats includingREALAUDIO®, REALVIDEO®, REALMEDIA®, APPLE QUICKTIME®, MICROSOFT WINDOWS®MEDIA FORMAT, QUICKTIME®, MPEG-2 LAYER III AUDIO, and MP3®. Typically,media files are designated with extensions (suffixes) indicatingcompatibility with specific formats. For example, media files (e.g.,audio and video files) ending in one of the extensions, ram, .rm, .rpm,are compatible with the REALMEDIA® format. Some examples of fileextensions and their compatible formats are listed in the followingtable. A more exhaustive list of media types, extensions and compatibleformats may be found at http://www.bowers.cc/extensions2.htm. FormatExtension REALMEDIA ® .ram, .rm, .rpm APPLE QUICKTIME ® .mov, .qifMICROSOFT .wma, .cmr, .avi WINDOWS ® MEDIA PLAYER MACROMEDIA FLASH .swf,.swl MPEG .mpg, .mpa, .mp1, .mp2 MPEG-2 LAYER III Audio .mp3, .m3a, .m3u

Metadata, literally means “data about data.” Metadata is data thatcomprises information that describes the contents or attributes of otherdata (e.g., media file). For example, a document entitled, “Dublin CoreMetadata for Resource Discovery,” (http://www.ietf.org/rfc/rfc2413.txt)separates metadata into three groups, which roughly indicate the classor scope of information contained therein. These three groups are: (1)elements related primarily to the content of the resource, (2) elementsrelated primarily to the resource when viewed as intellectual property,and (3) elements related primarily to the instantiation of the resource.Examples of metadata falling into these groups are shown in thefollowing table. Content Intellectual Property Instantiation TitleCreator Date Subject Publisher Format Description Contributor IdentifierType Rights Language Source Relation Coverage

Sources of metadata include web page content, uniform resource locators(URLs), media files, and transport streams used to transmit media files.Web page content includes HTML, XML, metatags, and any other text on theweb page. As explained in more detail, herein, metadata may also beobtained from the URLs the web page, media files, and other metadata.Metadata within the media file may include information contained in themedia file, such as in a header or trailer, of a multimedia or streamingfile, for example. Metadata may also be obtained from the media/metadatatransport stream, such as TCP/IP (e.g., packets), ATM, frame relay,cellular based transport schemes (e.g., cellular based telephoneschemes), MPEG transport, HDTV broadcast, and wireless based transport,for example. Metadata may also be transmitted in a stream in parallel oras part of the stream used to transmit a media file (a High Definitiontelevision broadcast is transmitted on one stream and metadata, in theform of an electronic programming guide, is transmitted on a secondstream).

FIG. 1 is a block diagram illustrating a system, generally designated100, in accordance with an exemplary embodiment of the presentinvention. The system 100 includes a plurality of server computers 18,20, a plurality of user computers 12, 14, and a plurality of databases21, 22. The server computers 18, 20 and the user computers 12, 14 may beconnected by a network 16, such as for example, an Intranet or theInternet. The user computers 12, 14 may be connected to the Intranet orInternet by a modem connection, a Local Area Network (LAN), cable modem,digital subscriber line (DSL), or other equivalent coupling means.Alternatively, the computers communicate through a communicationsnetwork by a cable, twisted pair, wireless based interface (cellular,infrared, radio waves) or equivalent connection utilizing data signals.Databases 21, 22 may be connected to the user computers and the servercomputers by any means known in the art. Databases may take the form ofany appropriate type of memory (e.g., magnetic, optical, etc.).Databases 21, 22 may be external memory or located within the servercomputer or the user computer. Each user computer 12, 14 preferablyincludes a video display device for displaying information and a browserprogram (e.g. MICROSOFT INTERNET EXPLORER®, NETSCAPE NAVIGATOR®, etc.),as is well known in the art.

Computers may also encompass computers embedded within consumer productsand other computers. For example, an embodiment of the present inventionmay comprise computers (as a processor) embedded within a television, aset top box, an audio/video receiver, a CD player, a VCR, a DVD player,a multimedia enable device (e.g., telephone), and an Internet enableddevice.

In an exemplary embodiment of the invention, the server computers 18, 20include one or more program modules and one or more databases whichallow the user computers 12, 14 to communicate with the server computer,and each other, over the network 16. The program module(s) of the servercomputers 18, 20 include program code, written in PERL, ExtensibleMarkup Language (XML), Java, Hypertext Mark-up Language (HTML), or anyother equivalent language which allows the user computers 12, 14 toaccess the program module(s) of the server computer through the browserprograms stored on the user computers. Although only two user computers12, 14, two server computers 18, 20, and two databases 21, 22 arelabeled in FIG. 1, those of ordinary skill in the art will realize thatthe system 100 may include any number of user computers, servercomputers, and databases.

In an exemplary embodiment of the present invention, media files andrelated metadata are searched for and retrieved by reading, extracting,enhancing, and grouping metadata describing the contents of files. FIG.2 is a flow diagram of an exemplary search and retrieval process inaccordance with the present invention. Discovery (step 24) comprises anautomated process referred to as a spider or web crawler, for searchingweb sites or data available through a communications network. Each website may comprise any number of web pages and/or data on storage devices(hard drives, flash cards, disc drives, optical disc storage). Thespider utilizes predetermined algorithms to continuously search formedia files on web pages and file directories at each searched web site.The spider also searches each web site for links to other web sites,unique streams, and downloadable files.

Upon finding a media file, metadata associated with that file isextracted (step 26). Metadata is extracted from sources such as the nameof the media file, the MIME responses, links to the media file, textsurrounding the media file on the website, metatags (descriptiveinformation embedded in sources as program code or HTML) in or surroundthe media file, content partners supplying metadata about their files,and the results of reading the metadata of the media file with aninterpretive extraction process.

Extracted metadata is enhanced in step 28. The extracted metadataassociated the media files are stored in memory (e.g., transferred to adatabase). The metadata is assessed, analyzed, and organized inaccordance with attributes associated with the media file. If metadatainformation is missing from the extracted metadata, it is added (step28). If metadata information is incorrect, it is corrected (step 28).For example, if metadata associated with a song comprises the fields ofComposer, Title, Musician, Album Name, and Music Genre, but is missingthe date the song was copyrighted, the copyright date is added to theextracted metadata. The metadata (e.g., copyright date) used to enhancethe extracted metadata is obtained from at least one of several sources.These sources include a baseline database of metadata associated withthe search target (e.g., the particular song of interest) and thesemantic and technical relationships between the fields in the extractedmetadata.

The extracted metadata, which may be enhanced, is categorized inaccordance with specific metadata attributes in step 30. At this pointthe links, e.g., uniform resource indicators (URLs) in the form ofuniform resource locators (URLs) for web pages and data files, may betransferred to the user, the URL of the media file may be transferred tothe user, or the categorized metadata may be used (e.g., transferred toa search engine) to search and retrieve the target media file. In anexemplary embodiment of the invention, the target streaming media streamautomatically starts playing. For example, a specific song is searchedfor, and the ultimate result is the playing of the song on the user'scomputer system.

Uniform resource indicators (URIs) are a universal set of names thatrefer to existing protocols or name spaces that identify resources(e.g., website, streaming media server), services (e.g., videos ondemand, internet radio), devices (e.g., mobile phone, internet enableappliance), and data files (e.g., media files and text documents). A URLis a form of a URL that expresses an address that maps to an accessalgorithm using network protocols (e.g., TCP/IP or a MPEG transportscheme). When a URL is used, a specific resource, service, device, ordata file may be accessed and/or manipulated. An alternative form of aURI known as an Internet protocol number or address (IP) is a series ofnumbers that refers to a specific resource, service, or data file.Optionally, a URL is mapped to an IP number, which provides two ways toaccess a desired resource (e.g., a resource is accessed either by usingwww.whitehouse.gov or the IP address 198.137.240.91).

FIG. 3 is a functional block diagram of an exemplary search andretrieval system, designated 300, in accordance with the presentinvention. System 300 comprises a plurality of autonomous, interactingagents for collecting, extracting, enhancing, and grouping mediametadata. Although system 300 depicts the agents performing in anexemplary order, agents may perform respective functions in any order.Each agent receives and provides data from and to data queues. Dataresiding on a data queue is available to all agents. In an exemplaryembodiment of the invention, media files and associated metadata arestored in memory (e.g., a database) and assigned an identifier (id). Theids are enqueued, and the agents receive and provide the ids from and tothe queues. Agents retrieve associated data (e.g., metadata) from memoryto perform various functions, and store the processed data in memory(e.g., update the database).

Spider 66 incorporates a process of seeding to search for media andrelated files. FIG. 4 is a flow diagram of an exemplary spider seedingprocess in accordance with the present invention. The spider is seededin step 36. The spider seeds its search by adding terms that are relatedto the query being used to index media. Additionally, the spider addsmedia related terms to the search, such as “MP3” and “Real Audio”.Adding media related terms to the search tend to limit the search tomedia related files and URIs (in the form of links). For example, addingstreaming media related terms to the search tends to limit the search tostreaming media related files and links. The spider receives the searchresults and uses the links to perform more searches. The input queue ofthe spider may be seeded with several types of information, such as theresults of querying other search engines, manually generated sets of webpage URLs, and processing proxy cache logs (i.e., web sites that otherusers have accessed).

The spider uses seed URLs to search (step 38) and retrieve (step 40) theHTML text from located web sites. The file name and MIME type of the website are stored in memory. The text is parsed to look for links to otherweb resources associated with media in step 42. The HTML code of eachweb page is examined for anchor tags, form elements, known JavaScriptfunctions, etc., to find other web resources associated with media.These newly found web resources are used as seeds for the spider foradditional searches (added to the spider input queue) by repeating steps36 through 42 using the new seeds.

Referring again to FIG. 3, the parsed results (from step 42 in FIG. 4)relating to the media are passed to extraction agent 68 via anextraction queue 67. Results not associated with the media are notpursued. The extraction queue 67 comprises URLs to be analyzed withrespect to associated media metadata. The extraction queue 67 maycomprise metadata queue entries such as media URLs, Web page URLs, Webpage titles, Web page keywords, Web page descriptions, media title,media author, and media genre. Each queue entry added to the extractionqueue is assigned a processing time and a priority. In an exemplaryembodiment of the invention, each queue entry is given a processing timeof “now” and the same default priority. The iterative seeding processincreases the number of queue entries added to the extraction queue 67.

The extraction agent 68 comprises an interpretive metadata extractor anda database retriever. The extraction agent 68 distributes and performsenhanced metadata extraction of queue entries on the extraction queue67. FIG. 5 is a flow diagram of an exemplary distribution and extractionprocess in accordance with the present invention. Queue entriescontained in the extraction queue 67 are dequeued and distributed tomedia specific extractors in step 46. The extraction queue entries aredequeued and distributed in priority and time order. Preferably, thefile extension, MIME type, and/or file identification for each queueentry is examined to determine the type of media format. The queue entryis than sent to the appropriate media specific extractor. Optionally,other types of data are used to determine the media format of a file(for example, the extraction queue 67 reads the metadata embedded in amedia file to determine that it is a Real Media video file).

In step 48, queue entries sent to a specific media extractor areextracted by that specific extractor. Metadata extraction comprises theprocess of extracting metadata from the media file or from related mediacontent (e.g., from the referring web page). Types of media specificextractors include multimedia and streaming extractors that can extractmetadata from formats such as REALAUDIO®, REALVIDEO®, REALMEDIA®, APPLEQUICKTIME®, MICROSOFT WINDOWS® MEDIA FORMAT, QUICKTIME®, MPEG-2 LAYERIII AUDIO, and MP3®, for example. As discussed in more detail herein,interpretive metadata extraction captures and aggregates metadatapertaining to a media file using metadata from the media stream, thirdparty databases, the referring web page, and the URL, and replacesinaccurate metadata with known good metadata. An Internet stream objectis created comprising the media file from the URL, metadata extractedfrom the media file and an identifier (id). The Internet stream objectis automatically stored in memory (step 50). In an exemplary embodimentof the invention, memory storage comprises providing the object to arelational database management system (DBMS) for storage and databasemanagement.

In step 52, it is determined if the accessible media file and theassociated metadata links are valid. Validation comprises determining ifthe Web page comprises a link to a desired media file, and alsodetermining if the desired media file works. In an exemplary embodimentof the invention, a streaming media file is retrieved and played todetermine it is valid. If determined to be invalid (not successful instep 52), the Internet stream object is assigned a later time andpriority. The Internet stream object is requeued to the extractor, andsteps 48 through 50 are repeated with at the later time and inaccordance with the newly assigned priority. If extraction is valid(successful in step 52), the Internet streaming object is queued andavailable to all agents.

Extraction agent 68 captures and aggregates media specific metadatapertaining to the media (including multimedia and streaming media) fromsources such as the media URL, the referring Web page URL, title, keywords, description, and third party databases. FIG. 6 is a flow diagramof an exemplary interpretive metadata extraction and database retrievalprocess in accordance with the present invention. Metadata, which may beinaccurate or “noisy,” is extracted (step 60), parsed and indexed (step62), compared with fields in known databases (step 64), and replaced(step 65) with accurate metadata obtained from a valid (ground truth)database. Metadata is indexed and parsed into metadata fields (step 62)and compared to other databases (step 64), such as a music or moviedatabase, whose accuracy is known (ground truth databases). Ground truthdatabases may be obtained from sources such as third party databases,previously created databases, and user entered databases, for example.Noisy fields are corrected and/or replaced with accurate data (step 65).New fields are added if appropriate (step 65).

For example, assume the spider 66 finds a media file containing a musicsong. The metadata is extracted by extracting agent 68, and parsed andindexed into the following metadata fields: the referring URL, the mediaURL, the title, and the performer of the song. The information containedin these fields is as follows. FIELD CONTENTS The referring URLwww.singingfish.com/index.html Media URL www.singingfish.com/foobar.RAWTitle “I am the Fishman” Performer Paul McCarpney

The metadata fields are compared to a known database, such as a thirdparty database, to compare contents of the metadata fields with thecontents of the fields in the known database. In this example, assume aknown database is located and contains the following indexed metadata.FIELD CONTENTS Copyright 1984 Title “We are the Fishmen” Album RubberSuit Music Genre Light Rock Performer John Lennon Performer PaulMcCarpney

The interpretive extraction agent 68, adds the missing fieldscorresponding to the copyright, album, music genre, and composer, andadds the additional performer (i.e., John Lennon). The interpretiveextraction 68 also corrects the title of the song from “I am theFishman” to “We are the Fishmen” because the database comprises valid orauthoritative metadata. Thus, prior to these enhancements, this mediafile could only be located if a user enter “Paul McCarpney” as theperformer and/or “I am the Fishman” as the title. As a result of theenhancements provided by the interpretive metadata extraction agent 68,a user may locate this media file also by searching for any of theresultant fields (e.g., the album name or the composer).

Not all databases queried are determined to be ground truth databases.FIG. 7 is a flow diagram of an exemplary process for querying databasesin accordance with the present invention. Noisy metadata (metadata thatneeds to be cleaned up before being processed) are compared to potentialground truth databases to determine if a potential ground truth databasequalifies as a ground truth database. In step 84, noisy metadata in eachfield (e.g., artist, title, album) is separated into keywords bytransforming any connecting characters into white space. For example,“oops_did_it_again” is transformed to the cleaned up “oops i did itagain”. Connecting characters may include, for example, period (“.”),underscore (“_”), backslash (“\”), forward slash (“/”), comma (“,”),asterisk (“*”), hyphen (“-”), and/or any other appropriate connectingcharacter. The fields in the noisy metadata are used to perform afull-text query against one or more fields in the potential ground truthdatabases (step 86).

A score is calculated, in step 88, to quantify the degree of similaritybetween the noisy data (candidate metadata) and potential ground truthdata (valid metadata). In an exemplary embodiment of the invention, thenumber of matching keywords in the fields being compared determines ascore. For example, if the input query is “oops i did it” and twopotential ground truth data records are “oops i did it again” and “didit again for you”, the first score is 4 and the second score is 2. In analternate embodiment of the invention, fields are also assigned weights,which are multiplied by the number of matching keywords. For example,the artist field may be assigned a weight of 3, and the copyright datefield may be assigned a weight of 1. Thus, if two keywords match in eachof the artist and copyright fields, the score for the artist field is 6,and the score for the copyright field is 2. Further, individual fieldscores may be added, averaged, or combined by any appropriate means toderive a cumulative database score. The scores are compared to athreshold value (step 90) to determine if the potential ground truthdatabase qualifies (step 92), or does not qualify (step 94) as a groundtruth database. If a database qualifies as a ground truth database, itis used by the interpretive extraction process as described herein. Thethreshold value may be predetermined and constant, or may be adaptivelydetermined in accordance with the range of calculated scores.

Referring again to FIG. 3, the validator 72 dequeues entries from thequeue in time and priority order. The validator 72 validates the mediadata by determining if the Web page comprises a link to a desired mediafile and also determining if the desired media file works. Validation isperformed at a future point in time (e.g., check if the URL is stillalive in 3 days), or alternatively, at periodic future points in time.If validity changes from valid to invalid, a notification is sent topromoter 82, as indicated by arrow 70. Validity may change from valid toinvalid, for example, if the media file was removed from the linkingURL.

The virtual domain detector 74 dequeues data from the queue in time andpriority order. The virtual domain detector 74 looks for duplicatedomains (field of the URL). If duplicates are found, they are identifiedas such and queued. The queued ids are available to all agents.

It is not uncommon for Web pages and multiple servers with differentportions of a URL, e.g., different domains, to host the same mediacontent. Further, the same media content may be available in differentformats and bit rates. The grouper 76 analyzes and compares URLs in thedatabase. The grouper 76 combines variants of the same media URL andcreates a group in which all metadata for similar URLs are added to thegroup. URLs are analyzed to determine if they are variations of relatedfiles. For example, if two URLs share a very complex path that differsonly in the file extension, the two URLs are considered to be related.Differences are eliminated by masking out tokens at the same relativelocation with respect to the original string.

FIG. 8 is a flow diagram of an exemplary grouping process in accordancewith the present invention. Grouping comprises the steps of binning 102and iterative masking 104. Binning 102 comprises the steps of selectingand sorting URLs (step 106) and combining URLs having common specifiedattributes into bins (step 108). In step 106, each URL in the databaseis analyzed to determine the contents of specific fields. URLs havingsimilar contents in the specified fields are placed (binned) into commonsets or bins of URLs (step 108). All URLs in the database are placedinto bins. Each bin has a smaller number of URLs than the number of URLsin the database. Although, it is possible that all URLs in the databaseare placed into the same bin, it is highly unlikely. As a result of thebinning process 102, each bin comprises at least one URL, and the URLscontained in bins comprising a plurality of URLs have at least onecommon attribute (i.e., same content in specified field(s)). Examples ofspecified fields include fields indicating artist, linking URL, title,copyright, host URL, duration, bit rate, sampling rate, etc. In anexemplary embodiment of the invention, URLs are binned if they have thesame content for the fields indicating host URL and duration. Oneadvantage of binning is that the number of URLs contained in a bin isrelatively small compare to the number of URLs contained in thedatabase, thus partitioning the URLs in the database into moremanageable sets of URLs.

Selected bins are iteratively masked in step 104. The masking process104 is performed on URLs on a bin by bin basis. Each field of each URLis compared to a mask. Not all bins require processing by the iterativemasking process 104. In an exemplary embodiment of the invention, binscontaining only a single URL are not iteratively masked 104, and binscontaining a plurality of URLs are processed in accordance with theiterative masking process 104.

FIG. 9 is a flow diagram of an exemplary iterative masking process inaccordance with the present invention. Iterative masking (step 104)comprises creating a “mask” (step 110) and comparing the mask with eachURL in a bin (step 112). A mask comprises at least one character to beremoved from the contents of a field within a URL. In an exemplaryembodiment of the invention, a mask is a string of characters. Forexample, a mask may comprise a string of characters pertaining to bitrate of the streaming media content, formatting of the streaming media,or any related characteristic. The mask is compared to each field in aURL in a bin, in step 112. It is determined if any of the characters inthe mask match characters in the URL (step 114). If a match exists, thematching character, or characters, is removed from the URL (step 116),otherwise the URL is unchanged. This process is repeated until all URLsin the bin have been compared with the mask (step 118).

Resultant URLs (i.e., URLs that have been compared to the mask) in thebin are compared and collapsed into a single URL if they are the same(step 120). For example, if four URLs differ only by bit rate, and thebit rate of each URL is masked out, the resulting four URLs arecollapsed into a single URL. If more bins have been selected, theiterative masking process is repeated for the next bin (step 122)starting at step 112. Grouped URLs are queued and available for allagents.

For example, assume all URLs in the database have been binned such thatall URLs comprising the same referring URL are binned together. Thus,assume the following URLs are in the same bin.

-   -   http://foo.bar.com/video/someArtist/myVideo_(—)28.ram    -   http://foo.bar.com/video/someArtist/myVideo_(—)56.ram    -   http://blatz.com/56/someArtist/yourVideo.ram    -   http://blatz.com/28/someArtist/yourVideo ram        Further assume that the mask is a string of characters related        to bit rate including 28, 56, and 100. Applying this mask to the        above URLs and removing the matched characters results in the        following URLs.    -   http://foo.bar.com/video/someArtist/myVideo_.ram    -   http://foo.bar.com/video/someArtist/myVideo_.ram    -   http://blatz.com//someArtist/yourVideo.ram    -   http://blatz.com//someArtist/yourVideo.ram        Instead of the bin containing four unique URLs, the bin now        contains two copies each of two unique URLs. Each of the two        copies is collapsed into a single URL, resulting in the        following URLs.    -   http://foo.bar.com/video/someArtist/myVideo_.ram    -   http://blatz.com//someArtist/yourVideo.ram

Referring again to FIG. 3, metadata quality improver 78 dequeues entriesin time and priority order. Metadata quality improver 78 enhancesmetadata by adding fields of metadata based upon the contents of thefields in the URL of the media file and the contents of the fields inthe URL of the referring Web page. The media file is then searchableunder the subject heading of the added metadata. For example, astreaming media file may have a referring Web page at www.cnn.com. Themetadata quality improver 78 adds the term “news” to the metadataassociated with the streaming media file, because cnn is related tonews. As a result, the streaming media file is now searchable under thesubject heading of “news”. Expert based rules are used to associatefield contents with metadata. Metadata quality improver 78 applies rulesto eliminate duplicate URLs that point to the same data, rules tocollect variants of media files with the same content but differentencodings or formats (e.g., for multimedia and streaming media), andrules to update metadata fields using prefix URL associations. Themetadata quality improvement process comprises prefix rule evaluation,genre annotation, and MUZE® (a commercial database containing metadataabout music including song title, music author, and album information)annotation.

FIG. 10 is a flow diagram of an exemplary metadata quality improvementprocess comprising prefix rule evaluation, genre annotation, and MUZE®annotation in accordance with the present invention. Prefix ruleevaluation comprises reorganizing the fields in the media URL anddetermining if an association exists between known sets of metadata andthe first field content. Genre annotation comprises updating the genremetadata to ensure proper formatting. MUZE® annotation comprises editingfields of the metadata to improve the quality of other fields of themetadata.

The fields of the URL are reorganized in step 138. In an exemplaryembodiment of the invention, the URL is reorganized in reverse order.Thus the first field of the URL becomes the last field and the lastfield becomes the first. In many instances, this results in areorganized URL having its most specific field first and its leastspecific field last. In many instances, this also results in the firstset of contiguous fields (i.e., prefix) of the reorganized URL havingassociated metadata. The first field of the reorganized URL is analyzedto determine if an association exists between the first field andpredetermined sets of metadata (step 140). Predetermined sets ofmetadata may comprise metadata obtained from other fields in themetadata and/or terms (metadata) contained in a database of terms. If itis determined that an association exists (step 142), the associatedmetadata are added to the original metadata in step 148. After metadataare added, it is determined if the reorganized URL contains more fields(which have not been analyzed for associated metadata) in step 150. Ifno associated metadata are identified (step 140 and 142), it is alsodetermined if more fields exist (step 150). If more fields exist, thenext field is analyzed to determine if an association exists between thenext field and the predetermined sets of metadata (step 146). In anexemplary embodiment of the invention, the next field is the nextcontiguous field. If no associated metadata are identified (step 142),no new metadata are added to the metadata associated with the mediafile. If associated metadata are identified, the associated metadata areadded to the original metadata in step 148. This process is continueduntil all the fields in the reorganized URL have been analyzed. At thispoint, metadata associated with the longest match (i.e., the greatestnumber of fields having associated metadata) have been added to theoriginal metadata. Databases are updated with the newly added metadata,and the associated ids are queued and available to all agents.

In an exemplary embodiment of the invention, the genre metadata ifupdated to ensure proper formatting and correctness. The updatedmetadata is analyzed to determine if the genre field(s) are correct. Ifit is determined that the genre field(s) are not correct, they areupdated. The genre fields are updated in accordance with predeterminedassociation rules. For example, assume the contents of a fieldpertaining to category is “music” (i.e., “category=music”). The metadatais analyzed, and the metadata terms “artist=Freddy Roulette”,“genre_MP3=punk rock” are found. The field associated with category willbe changed from music to punk rock, resulting in “category=punk rock”.In this example, the category field is changed because a predeterminedassociation rule is encoded to change the “category” field to the sameas the “genre_MP3” field.

In another exemplary embodiment of the invention, the iterative processis halted after metadata associated with the longest prefix ofcontiguous fields of the reorganized URL are identified, and metadataassociated with the prefix, and not the individual fields is added tothe original metadata. For example, assuming a URL has ten fields, ifthe first four fields of the reorganized URL have associated metadata,and the fifth field does not have associated metadata, the sixth throughthe tenth fields are not analyzed for associated metadata. In thisexample, the metadata associated with the first four fields, i.e., theprefix, and not the individual fields, (and, as will be explained hereinwith reference to muze annotation, possibly the metadata associated withthe fifth field) are added to the original metadata.

Metadata is added to the metadata associated with the media file. Addedmetadata may comprise metadata corresponding to category, title,delivery mode, publisher, performer, program title, creation country,and language, for example. The added metadata may be in the form oftextual data (e.g., new terms) and/or URLs (e.g., new links). Also, inaccordance with the MUZE® annotation rule, added metadata may comprisethe content of the field in the reorganized URL that is next to thematching prefix (e.g., first non-matching field). The content of thefield is edited to replace connecting characters with spaces, and thenadded as new metadata. Connecting characters may include, for example,period (“.”), underscore (“_”), backslash (“\”), forward slash (“/”),comma (“,”), asterisk (“*”), hyphen (“-”), and/or any other appropriateconnecting character. This muze annotation rule is advantageous for URLscomprising field content of “MP3”. In an exemplary embodiment of theinvention, all reorganized URLs beginning with the prefix“com.MP3.downloads” are categorized as music and the recommended title(i.e., song title) is based on the filename as given in the field nextto the matched prefix. For example, assume the following reorganizedURL, “com.MP3.downloads/Freddy Roulette/Laundry_Mat_Blues”. In thisexample, Freddy Roulette is the content of the artist field andLaundry_Mat_Blues is the content of the title field. The metadataquality improvement process finds a match for the prefix of thereorganized URL ending with Freddy Roulette. Because the reorganized URLbegins with com.MP3, the metadata quality improver 78 edits the nextfield (i.e., Laundry_Mat_Blues) after the matched prefix and adds theedited data as the recommended title of the song. The edited fieldcontent has no underscores. Thus the resultant added metadata in thisexample is “Laundry Mat Blues”. Databases are updated with the newlyadded metadata, and the associated ids are queued and available to allagents. Examples of the types of metadata that are added to matchedfields are shown in the following table. Field Prefix Added Metadataorg.npr.www/ramfiles/atc Category: Radio Delivery Mode: BroadcastPublisher: NPR Performer: Noah Adams Program Title: All ThingsConsidered Language: English com.sportsline.www/ Category: Sportsu/audio/basketball/nba Genre: Basketball Creation Country: US Language:English com.msnbc.www Category: News Recommended Title: Referring PageTitle com.mp3 Category: Music com.mp3.downloads Category: MusicRecommended Title: Filename in the next field of the URL (i.e., textafter the matched prefix)

The full-text relevancy ranker 80 comprises ranking and sorting data(e.g., media metadata) based on a variety of semantic and technical datafields (e.g., title, author, date, duration, bit rate, etc.). Full-textrelevancy ranker 80 is depicted as part of the work flow architecture ofsystem 300. This depiction is exemplary. In another embodiment of theinvention, full-text relevancy ranker 80 is not part of the workflowarchitecture. The option to include full-text relevancy ranker 80 aspart of the workflow architecture (or not) is depicted by the dashedarrows in FIG. 3 (from metadata quality improver 78 to full-textrelevancy ranker 80, from full-text relevancy ranker 80 to promoter 82,and from metadata quality improver 78 to promoter 82). FIG. 11 is a flowdiagram of an exemplary full-text relevancy ranking process inaccordance with the present invention. Media metadata describing thesemantics of the content are sorted and grouped into broad categories(e.g., who, what, where, when) in step 156. For example, artist of astreaming media file, type of streaming media, date the streaming mediawas created, and creation location of the streaming media. These broadcategories are individually weighted along with technical parameterssuch as bit rate, duration, fidelity (audio sampling rate), etc., instep 158. A relevance score is calculated for each URL in accordancewith associated weights in step 160. The relevancy score is based uponseveral weighting criteria, such as the number of times a query termoccurs in the metadata (term frequency), the number of links to thereferenced Web site, number of terms between query terms in the text forthe metadata, and the file type selected for a search (e.g., wav, MP3,ram, wma).

For example, suppose a user enters a search query comprising the terms“Mozart”, “Magic Flute”, and “Red”. The full-text relevancy ranker 80,knowing that Mozart is a name of a composer (encoded rule), semanticallyassociates Mozart with the who category and looks for “Mozart” in afield designated as WhoCreation. Magic Flute is recognized as a musiccomposition and is semantically associated with the What category andlooked for in the Title field. Weights of greater value are assigned toterms that are associated with semantic categories than to terms thatare not associated with semantic categories. Thus, matches to “Mozart”and “Magic Flute” are assigned a greater weight, and accordingly ahigher relevancy score, than the unrelated term “Red”. The full-textrelevancy ranker 80 also considers technical parameters in thecalculation of the relevancy score. In the current example, if the termnews were added to the search query, the full-text relevancy ranker 80looks for news pieces about Mozart and the Magic Flute, rather than fora piece of music. In an exemplary embodiment of the invention, full-textrelevancy ranker 80 searches for news articles by considering theduration of the indexed files. For example, the full-text relevancyranker 80 knows that news pieces typically have a shorter duration thanmusic files (an encoded rule). Accordingly, the full-text relevancyranker 80 assigns a higher score to files with shorter lengths. Iffull-text ranker 80 is incorporated as part of the workflowarchitecture, the database is updated with the full-text relevancyranked data and the associated ids are queued and available to allagents. If full-text relevancy ranker 80 is not incorporated as part ofthe workflow architecture, no associated ids are queued and madeavailable to all agents. Rather, the results are made directly availableto search systems and/or users.

Referring again to FIG. 3, the Promoter 82, formats and prioritizes datafor a target search system (e.g., search engine). Promoter 82 adds,deletes, and/or updates the data (including metadata) associated with amedia file in accordance with the requirements of the target searchsystem. Promoter 82 also provides an indication to the search system ofthe trustworthiness of the media data. In an exemplary embodiment of thesystem, trustworthiness is determined in accordance with predeterminedencoded rules. For example, promoter 82 may determine that metadataassociated with the title fields is the most trustworthy, and thatmetadata associated with the genre fields is less trustworthy. Thishierarchy of trustworthiness is provided to the target search system ina compatible format. The target search system may then use thishierarchy of trustworthiness to conduct its search or pass the URLs onto the user.

The present invention may be embodied in the form ofcomputer-implemented processes and apparatus for practicing thoseprocesses. The present invention may also be embodied in the form ofcomputer program code embodied in tangible media, such as floppydiskettes, read only memories (ROMs), CD-ROMs, hard drives, high densitydisk, or any other computer-readable storage medium, wherein, when thecomputer program code is loaded into and executed by a computer, thecomputer becomes an apparatus for practicing the invention. The presentinvention may also be embodied in the form of computer program code, forexample, whether stored in a storage medium, loaded into and/or executedby a computer, or transmitted over some transmission medium, such asover electrical wiring or cabling, through fiber optics, or viaelectromagnetic radiation, wherein, when the computer program is loadedinto and executed by a computer, the computer becomes an apparatus forpracticing the invention. When implemented on a general-purposeprocessor, the computer program code segments configure the processor tocreate specific logic circuits.

The present invention may be embodied to update or replace the metadatarelating to a media file, contained in a database, web page, storagedevice, media file (header or footer), URI, transport stream, electronicprogram guide, and other sources of metadata, by using the sameprocesses and/or apparatuses described wherein.

Although the present invention is described in terms of exemplaryembodiments, it is not limited thereto. Rather, the appended claimsshould be construed broadly, to include other variants and embodimentsof the invention, which may be made by those skilled in the art withoutdeparting from the scope and range of equivalents of the invention.

1-18. (canceled)
 19. A computer-implemented method for ranking searchresults for multimedia items, the method comprising: providing aplurality of categories relating to the semantics of multimedia content;for each of a plurality of multimedia items: identifying metadataassociated with the multimedia item, the metadata describing a semanticcharacteristic of the multimedia item; and for each identified metadataassociated with the multimedia item, categorizing the identifiedmetadata into one of the plurality of categories relating to thesemantics of multimedia content; receiving a search query comprising atleast one search term; for each search term: semantically associatingthe search term with one of the plurality of categories relating to thesemantics of multimedia content; and identifying a categorized metadatamatching the search term, the categorized metadata selected from theassociated one of the plurality of categories; identifying a candidatemultimedia item from the plurality of multimedia items, the candidatemultimedia item associated with at least one of the identifiedcategorized metadata; and determining a score for the identifiedcandidate multimedia item, wherein the score is based on the categoriesassociated with all of the identified categorized metadata matching thesearch terms in the search query.
 20. The method of claim 19 furthercomprising ranking the identified candidate multimedia item according tothe determined score.
 21. The method of claim 19 further comprisingassigning a weight to each of the search terms based on the categoryassociated with each search term.
 22. The method of claim 21, whereinthe score is additionally based on the weights assigned to the searchterms in the search query.
 23. The method of claim 19, wherein thecategories comprise a who category.
 24. The method of claim 23, whereinthe who category indicates an artist of the multimedia item.
 25. Themethod of claim 19, wherein the categories comprise a what category. 26.The method of claim 25, wherein the what category indicates a type ofthe multimedia item.
 27. The method of claim 19, wherein the categoriescomprise a where category.
 28. The method of claim 27, wherein the wherecategory indicates a creation location of the multimedia item.
 29. Themethod of claim 19, wherein the categories comprise a when category. 30.The method of claim 29, wherein the when category indicates a creationdate of the multimedia item.
 31. A computer-readable medium whosecontents cause a computer to rank search results for multimedia itemsby: providing a plurality of categories relating to the semantics ofmultimedia content; for each of a plurality of multimedia items:identifying metadata associated with the multimedia item, the metadatadescribing a semantic characteristic of the multimedia item; and foreach identified metadata associated with the multimedia item,categorizing the identified metadata into one of the plurality ofcategories relating to the semantics of multimedia content; receiving asearch query comprising at least one search term; for each search term:semantically associating the search term with one of the plurality ofcategories relating to the semantics of multimedia content; andidentifying a categorized metadata matching the search term, thecategorized metadata selected from the associated one of the pluralityof categories; identifying a candidate multimedia item from theplurality of multimedia items, the candidate multimedia item associatedwith at least one of the identified categorized metadata; anddetermining a score for the identified candidate multimedia item,wherein the score is based on the categories associated with all of theidentified categorized metadata matching the search terms in the searchquery.
 32. The computer-readable medium of claim 31, wherein themetadata comprises elements related to content of the multimedia item.33. The computer-readable medium of claim 31, wherein the metadatacomprises elements related to intellectual property rights associatedwith the multimedia item.
 34. The computer-readable medium of claim 31,wherein the metadata comprises elements related to instantiation of themultimedia item.